use predictable name for csiaddons object #695

Madhu-1 · 2024-10-29T12:31:19Z

This PR does the below changes

Use the deployment/Daemonset as the owner of the csiaddons instead of the pod
Use a predictable name which includes the nodeID, owner name, and the kind to ensure that we get unique names per resource also if the pod is restarting we will reuse the same resource name for the csiAddonsNode object. Using Resource Kind as csiAddonsNode object is a namespaced resource
Take care of the worst case where the resource char limit is not met.

Note: The reason for this change is that we are planning to use the csi-addons node object resources to store the metadata related to the fencing itself.

Madhu-1 · 2024-10-30T11:39:48Z

For pod backed by daemonset

[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get po
NAME                                            READY   STATUS             RESTARTS       AGE
csi-cephfsplugin-provisioner-584b99fb94-9ndhv   5/6     Running   24 (89s ago)   102m
csi-cephfsplugin-trtm5                          2/2     Running            0              111m
csi-rbdplugin-provisioner-6cdbf69784-zpj2v      5/6     Running   24 (76s ago)   102m
csi-rbdplugin-rbh4f                             3/3     Running            0              2s
rook-ceph-exporter-minikube-78569dcc4b-g5v9r    1/1     Running            0              5d21h
rook-ceph-mds-myfs-a-84f8c88f8f-km9hn           1/1     Running            0              5d21h
rook-ceph-mds-myfs-b-57c9888496-5g89q           1/1     Running            0              5d21h
rook-ceph-mgr-a-78d9b7bf4f-qqb94                1/1     Running            0              5d21h
rook-ceph-mon-a-f55458858-gpqfs                 1/1     Running            0              5d21h
rook-ceph-operator-5c49669f69-8z6v9             1/1     Running            0              5d21h
rook-ceph-osd-0-cb9f64744-cplvb                 1/1     Running            0              5d21h
rook-ceph-osd-prepare-minikube-zjk6w            0/1     Completed          0              5d21h
rook-ceph-tools-68bf47bc65-9fdnp                1/1     Running            0              5d21h

[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get csic get csiaddonsnode -oyaml
apiVersion: v1
items:
- apiVersion: csiaddons.openshift.io/v1alpha1
  kind: CSIAddonsNode
  metadata:
    creationTimestamp: "2024-10-30T10:59:20Z"
    finalizers:
    - csiaddons.openshift.io/csiaddonsnode
    generation: 2
    name: minikube-rook-ceph-daemonset-csi-rbdplugin
    namespace: rook-ceph
    ownerReferences:
    - apiVersion: apps/v1
      blockOwnerDeletion: true
      controller: true
      kind: DaemonSet
      name: csi-rbdplugin
      uid: 57944793-9ff3-4295-b699-d71fff7fc497
    resourceVersion: "84214"
    uid: 816e884b-4792-4b76-bc99-12b81f25c162
  spec:
    driver:
      endpoint: pod://csi-rbdplugin-rbh4f.rook-ceph:9070
      name: rook-ceph.rbd.csi.ceph.com
      nodeID: minikube
  status:
    capabilities:
    - service.NODE_SERVICE
    - reclaim_space.ONLINE
    - encryption_key_rotation.ENCRYPTIONKEYROTATION
    message: Successfully established connection with sidecar
    state: Connected
kind: List
metadata:
  resourceVersion: ""

[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc delete po/csi-rbdplugin-rbh4f
pod "csi-rbdplugin-rbh4f" deleted
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get po
NAME                                            READY   STATUS             RESTARTS         AGE
csi-cephfsplugin-provisioner-584b99fb94-9ndhv   5/6     Running   24 (2m36s ago)   103m
csi-cephfsplugin-trtm5                          2/2     Running            0                113m
csi-rbdplugin-f7st7                             3/3     Running            0                4s
csi-rbdplugin-provisioner-6cdbf69784-zpj2v      5/6     Running   24 (2m23s ago)   103m
rook-ceph-exporter-minikube-78569dcc4b-g5v9r    1/1     Running            0                5d21h
rook-ceph-mds-myfs-a-84f8c88f8f-km9hn           1/1     Running            0                5d21h
rook-ceph-mds-myfs-b-57c9888496-5g89q           1/1     Running            0                5d21h
rook-ceph-mgr-a-78d9b7bf4f-qqb94                1/1     Running            0                5d21h
rook-ceph-mon-a-f55458858-gpqfs                 1/1     Running            0                5d21h
rook-ceph-operator-5c49669f69-8z6v9             1/1     Running            0                5d21h
rook-ceph-osd-0-cb9f64744-cplvb                 1/1     Running            0                5d21h
rook-ceph-osd-prepare-minikube-zjk6w            0/1     Completed          0                5d21h
rook-ceph-tools-68bf47bc65-9fdnp                1/1     Running            0                5d21h
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get csiaddonsnode -oyaml
apiVersion: v1
items:
- apiVersion: csiaddons.openshift.io/v1alpha1
  kind: CSIAddonsNode
  metadata:
    creationTimestamp: "2024-10-30T10:59:20Z"
    finalizers:
    - csiaddons.openshift.io/csiaddonsnode
    generation: 3
    name: minikube-rook-ceph-daemonset-csi-rbdplugin
    namespace: rook-ceph
    ownerReferences:
    - apiVersion: apps/v1
      blockOwnerDeletion: true
      controller: true
      kind: DaemonSet
      name: csi-rbdplugin
      uid: 57944793-9ff3-4295-b699-d71fff7fc497
    resourceVersion: "84348"
    uid: 816e884b-4792-4b76-bc99-12b81f25c162
  spec:
    driver:
      endpoint: pod://csi-rbdplugin-f7st7.rook-ceph:9070
      name: rook-ceph.rbd.csi.ceph.com
      nodeID: minikube
  status:
    capabilities:
    - service.NODE_SERVICE
    - reclaim_space.ONLINE
    - encryption_key_rotation.ENCRYPTIONKEYROTATION
    message: Successfully established connection with sidecar
    state: Connected
kind: List
metadata:
  resourceVersion: ""

For pod backed by deployment

[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get po
NAME                                            READY   STATUS             RESTARTS       AGE
csi-cephfsplugin-provisioner-584b99fb94-9ndhv   5/6     Running   25 (43s ago)   106m
csi-cephfsplugin-trtm5                          2/2     Running            0              116m
csi-rbdplugin-f7st7                             3/3     Running            0              3m19s
csi-rbdplugin-provisioner-6cdbf69784-tqlz8      6/6     Running            0              18s
rook-ceph-exporter-minikube-78569dcc4b-g5v9r    1/1     Running            0              5d21h
rook-ceph-mds-myfs-a-84f8c88f8f-km9hn           1/1     Running            0              5d21h
rook-ceph-mds-myfs-b-57c9888496-5g89q           1/1     Running            0              5d21h
rook-ceph-mgr-a-78d9b7bf4f-qqb94                1/1     Running            0              5d21h
rook-ceph-mon-a-f55458858-gpqfs                 1/1     Running            0              5d21h
rook-ceph-operator-5c49669f69-8z6v9             1/1     Running            0              5d21h
rook-ceph-osd-0-cb9f64744-cplvb                 1/1     Running            0              5d21h
rook-ceph-osd-prepare-minikube-zjk6w            0/1     Completed          0              5d21h
rook-ceph-tools-68bf47bc65-9fdnp                1/1     Running            0              5d21h
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get csiaddonsnode
NAME                                                      NAMESPACE   AGE   DRIVERNAME                   ENDPOINT                                                          NODEID
minikube-rook-ceph-daemonset-csi-rbdplugin                rook-ceph   37m   rook-ceph.rbd.csi.ceph.com   pod://csi-rbdplugin-f7st7.rook-ceph:9070                          minikube
minikube-rook-ceph-deployment-csi-rbdplugin-provisioner   rook-ceph   79s   rook-ceph.rbd.csi.ceph.com   pod://csi-rbdplugin-provisioner-6cdbf69784-tqlz8.rook-ceph:9070   minikube
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get csiaddonsnode minikube-rook-ceph-deployment-csi-rbdplugin-provisioner -oyaml
apiVersion: csiaddons.openshift.io/v1alpha1
kind: CSIAddonsNode
metadata:
  creationTimestamp: "2024-10-30T11:35:43Z"
  finalizers:
  - csiaddons.openshift.io/csiaddonsnode
  generation: 1
  name: minikube-rook-ceph-deployment-csi-rbdplugin-provisioner
  namespace: rook-ceph
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: Deployment
    name: csi-rbdplugin-provisioner
    uid: c7791f00-ac40-48cf-967c-69595ce65039
  resourceVersion: "84752"
  uid: 262b6c46-8c54-424b-889f-7b64009938c4
spec:
  driver:
    endpoint: pod://csi-rbdplugin-provisioner-6cdbf69784-tqlz8.rook-ceph:9070
    name: rook-ceph.rbd.csi.ceph.com
    nodeID: minikube
status:
  capabilities:
  - service.CONTROLLER_SERVICE
  - reclaim_space.OFFLINE
  - network_fence.NETWORK_FENCE
  - volume_replication.VOLUME_REPLICATION
  - volume_group.VOLUME_GROUP
  - volume_group.DO_NOT_ALLOW_VG_TO_DELETE_VOLUMES
  - volume_group.LIMIT_VOLUME_TO_ONE_VOLUME_GROUP
  - volume_group.MODIFY_VOLUME_GROUP
  - volume_group.GET_VOLUME_GROUP
  message: Successfully established connection with sidecar
  state: Connected
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc delete po/csi-rbdplugin-provisioner-6cdbf69784-tqlz8
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get csiaddonsnode
NAME                                                      NAMESPACE   AGE    DRIVERNAME                   ENDPOINT                                                          NODEID
minikube-rook-ceph-daemonset-csi-rbdplugin                rook-ceph   38m    rook-ceph.rbd.csi.ceph.com   pod://csi-rbdplugin-f7st7.rook-ceph:9070                          minikube
minikube-rook-ceph-deployment-csi-rbdplugin-provisioner   rook-ceph   114s   rook-ceph.rbd.csi.ceph.com   pod://csi-rbdplugin-provisioner-6cdbf69784-tqlz8.rook-ceph:9070   minikube
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get po
NAME                                            READY   STATUS             RESTARTS         AGE
csi-cephfsplugin-provisioner-584b99fb94-9ndhv   5/6     Running   25 (2m36s ago)   108m
csi-cephfsplugin-trtm5                          2/2     Running            0                118m
csi-rbdplugin-f7st7                             3/3     Running            0                5m12s
csi-rbdplugin-provisioner-6cdbf69784-4xk2q      6/6     Running            0                17s
rook-ceph-exporter-minikube-78569dcc4b-g5v9r    1/1     Running            0                5d21h
rook-ceph-mds-myfs-a-84f8c88f8f-km9hn           1/1     Running            0                5d21h
rook-ceph-mds-myfs-b-57c9888496-5g89q           1/1     Running            0                5d21h
rook-ceph-mgr-a-78d9b7bf4f-qqb94                1/1     Running            0                5d21h
rook-ceph-mon-a-f55458858-gpqfs                 1/1     Running            0                5d21h
rook-ceph-operator-5c49669f69-8z6v9             1/1     Running            0                5d21h
rook-ceph-osd-0-cb9f64744-cplvb                 1/1     Running            0                5d21h
rook-ceph-osd-prepare-minikube-zjk6w            0/1     Completed          0                5d21h
rook-ceph-tools-68bf47bc65-9fdnp                1/1     Running            0                5d21h
[🎩︎]mrajanna@li-2cfbef4c-22d9-11b2-a85c-a3e4a93c405f rook $]kuberc get csiaddonsnode minikube-rook-ceph-deployment-csi-rbdplugin-provisioner -oyaml
apiVersion: csiaddons.openshift.io/v1alpha1
kind: CSIAddonsNode
metadata:
  creationTimestamp: "2024-10-30T11:35:43Z"
  finalizers:
  - csiaddons.openshift.io/csiaddonsnode
  generation: 2
  name: minikube-rook-ceph-deployment-csi-rbdplugin-provisioner
  namespace: rook-ceph
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: Deployment
    name: csi-rbdplugin-provisioner
    uid: c7791f00-ac40-48cf-967c-69595ce65039
  resourceVersion: "84979"
  uid: 262b6c46-8c54-424b-889f-7b64009938c4
spec:
  driver:
    endpoint: pod://csi-rbdplugin-provisioner-6cdbf69784-4xk2q.rook-ceph:9070
    name: rook-ceph.rbd.csi.ceph.com
    nodeID: minikube
status:
  capabilities:
  - service.CONTROLLER_SERVICE
  - reclaim_space.OFFLINE
  - network_fence.NETWORK_FENCE
  - volume_replication.VOLUME_REPLICATION
  - volume_group.VOLUME_GROUP
  - volume_group.DO_NOT_ALLOW_VG_TO_DELETE_VOLUMES
  - volume_group.LIMIT_VOLUME_TO_ONE_VOLUME_GROUP
  - volume_group.MODIFY_VOLUME_GROUP
  - volume_group.GET_VOLUME_GROUP
  message: Successfully established connection with sidecar
  state: Connected

nixpanic

Please move the added vendored packages to a separate commit for easier reviewing.

Maybe add something to the documentation docs/csiaddonsnode.md about it too. A paragraph about the Lifecycle of the CSIAddonsNode CRs?

nixpanic

Thanks!

sidecar/internal/csiaddonsnode/csiaddonsnode.go

Pull request has been modified.

black-dragon74

LGTM. Thank you!

This PR does the below changes * Use the deployment/Daemonset as the owner of the csiaddons instead of pod * Use predictable name which includes the nodeID, owner name and the kind to ensure that we get unique names per resources also if the pod is restarting we will reuse the same resource name for the csiAddonsNode object. Using Resource Kind as csiAddonsNode object is a namespaced resouces * Take care of worst case where resource char limit is not meet. Signed-off-by: Madhu Rajanna <[email protected]>

This commit includes the vendor changes for the predictable name Signed-off-by: Madhu Rajanna <[email protected]>

added a document for the lifecycle of the CSIAddonsNode object. Signed-off-by: Madhu Rajanna <[email protected]>

Madhu-1 force-pushed the rename-csi-addons branch 2 times, most recently from 6c4828f to 8a7299e Compare October 30, 2024 09:25

mergify bot added the vendor Pull requests that update vendored dependencies label Oct 30, 2024

Madhu-1 force-pushed the rename-csi-addons branch 2 times, most recently from ab4c3c5 to e37ef73 Compare October 30, 2024 09:36

Madhu-1 force-pushed the rename-csi-addons branch from e37ef73 to 686ccf9 Compare October 30, 2024 11:40

Madhu-1 marked this pull request as ready for review October 30, 2024 11:40

mergify bot requested review from black-dragon74, iPraveenParihar, nixpanic, Rakshith-R and yati1998 October 30, 2024 11:40

Madhu-1 force-pushed the rename-csi-addons branch from 686ccf9 to 22bd6b7 Compare October 30, 2024 11:51

nixpanic requested changes Oct 31, 2024

View reviewed changes

Madhu-1 force-pushed the rename-csi-addons branch from 22bd6b7 to 9c7da84 Compare October 31, 2024 10:54

Madhu-1 requested a review from nixpanic October 31, 2024 10:55

Madhu-1 force-pushed the rename-csi-addons branch 3 times, most recently from 0073cdc to d21ddb2 Compare October 31, 2024 11:01

nixpanic previously approved these changes Oct 31, 2024

View reviewed changes

black-dragon74 reviewed Nov 4, 2024

View reviewed changes

sidecar/internal/csiaddonsnode/csiaddonsnode.go Outdated Show resolved Hide resolved

Madhu-1 force-pushed the rename-csi-addons branch from d21ddb2 to 5db6cf8 Compare November 4, 2024 06:47

Madhu-1 requested review from black-dragon74 and nixpanic November 4, 2024 06:48

black-dragon74 approved these changes Nov 4, 2024

View reviewed changes

nixpanic approved these changes Nov 5, 2024

View reviewed changes

Madhu-1 added 2 commits November 5, 2024 10:32

vendor: vendor changes for the predictable name

97ab96b

This commit includes the vendor changes for the predictable name Signed-off-by: Madhu Rajanna <[email protected]>

doc: lifecycle of the CSIAddonsNode

22ce0cb

added a document for the lifecycle of the CSIAddonsNode object. Signed-off-by: Madhu Rajanna <[email protected]>

Madhu-1 force-pushed the rename-csi-addons branch from 5db6cf8 to 22ce0cb Compare November 5, 2024 10:32

mergify bot merged commit 56db575 into csi-addons:main Nov 5, 2024
15 checks passed

Madhu-1 mentioned this pull request Nov 6, 2024

csiaddons: add new RBAC ceph/ceph-csi-operator#166

Merged

6 tasks

rewantsoni mentioned this pull request Nov 11, 2024

csiaddons: add rbac permission for setting ownerRef ceph/ceph-csi-operator#171

Merged

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use predictable name for csiaddons object #695

use predictable name for csiaddons object #695

Madhu-1 commented Oct 29, 2024 •

edited

Loading

Madhu-1 commented Oct 30, 2024

nixpanic left a comment

nixpanic left a comment

black-dragon74 left a comment

use predictable name for csiaddons object #695

use predictable name for csiaddons object #695

Conversation

Madhu-1 commented Oct 29, 2024 • edited Loading

Madhu-1 commented Oct 30, 2024

For pod backed by daemonset

For pod backed by deployment

nixpanic left a comment

Choose a reason for hiding this comment

nixpanic left a comment

Choose a reason for hiding this comment

black-dragon74 left a comment

Choose a reason for hiding this comment

Madhu-1 commented Oct 29, 2024 •

edited

Loading